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ABSTRACT 

A national study of effective schools programs 
provides information on the extent and characteristics of these 
programs in the nation's school dist- cts and schools. A 
questionnaire was mailed to a random sample of 1,685 school district 
superintendents. Findings are representative of the approximately 
16,000 local school districts in the nation. This report is divided 
into eight appendices: (i) selected characteristics of effective 
schools cited in research literature; (2) five school characteristic 
in the definition of effective schools programs included in Public 
Law 100-297; (3) Chapter 2 Block Grant Program (six authorized local 
targeted assistance programs); (4) estimates and confidence interval 
for selected questionnaire responses; (5) effective schools programs 
m the 50 largest cities (school year 1987-88); (6) examples of SEA 
assistance to school districts and schocls for effective schools 
programs; (7) tables supporting figures in report text and 
supplementary tables; and (8) major contributors to this report. 
Included are 15 tables and 28 figures, (si) 
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United States 

General Accounting Office 

Washington, B.C. 20548 



Human Resources Division 
B-235384 

September 13, 1989 

The Honorable Augustus F. Hawkins 
Chairman, Committee on Education and Labor 
House of Representatives 

Dear Mr. Chairman: 

You requested that we conduct a national study of effective schools pro- 
grams to provide information on the extent and characteristics of these 
programs in the nation's school districts aad schools. In subsequent dis- 
cussions with your office, we agreed to (1) determine the number of 
school districts with effective schools programs, (2) identify common 
program characteristics and practices, (3) describe how school districts 
evaluate the effect of their programs on students' academic achieve- 
ment, and (4) discuss federal requirements for evaluating these pro- 
grams. You were particularly interested in the ability of school districts 
to provide insight on whether effective schools programs improved the 
academic achievement of low-income and minority children. 

The Hawkins-Stafford Elementary and Secondary School Improvement 
Amendments of 1988 (P.L. 100-297) authorized Chapter 1 compensatory 
education funds and Chapter 2 educational improvement block grant 
funds to be used for funding effective schools programs. As you know, 
an effective schools program is a school-based prograni developed to 
improve the academic achievement of all children in a school, regardless 
of socioeconomic background or ethnicity, and in particular education- 
ally deprived children.' Effective schools programs, like other school 
improvement efforts, have become increasingly common as the educa- 
tional reform movement has burgeoned during the 1980s. 

Effective schools programs seek to develop or improve on school charac- 
teristics that have been identified in research as associated with high 
student achievement. Public Law 100-297 specifies five such character- 
istics that these programs should seek to achieve. They are (1) strong 
and effective administrative and instructional leadership, (2) emphasis 
on the acquisition of basic and higher-order sidlls, (3) a safe and orderly 
school environment, (4) a climate of expectations that virtually all chil- 
dren can learn under appropriate conditions, and (5) continuous assess- 
ment of students and programs to evaluate the effects of instruction. 



*lhe Department of Education defines educationally de;irived children as children whose educational 
attainment is below the level that is appropriate for their age. 
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To collect the information in this report, we mailed copies of a question- 
naire to a random sample of 1,685 school district superintendents. Our 
findings are representative of the approximately 16,000 local school dis- 
tricts in the nation. The questionnaire we used asked superintendents 
about effective schools programs operating in their schools during 
school year 1987-88. This report elaborates on the briefing we provided 
congressional staff on June 20, 1989. 



To measure the overall number of districts and schools with effective 
schools programs, we asked district officials whether any of their 
schools operated school improvement programs based on the findings of 
effective schools research. District responses to our questionnaire indi- 
cate that about 41 percent or 6,500 of the nation's school districts had 
effective schools programs in operation in approximately 38,000 ele- 
mentary and secondary schools during school year 1987-88. Many dis- 
tricts reported that their programs had been established recently; over 
half had effective schools programs that were first implemented during 
school years 1986-87 or 1987-88. An additional 17 percent or about 
2,600 of the nation's districts hav^ plans to implement effective schools 
programs during school years 1988-89 or 1989-90. 

To provide the Congress with information on the extent to which effec- 
tive schools programs include specific activities recommended by pro- 
grairt experts, we measured the number of districts and schools with 
programs that met certain program criteria. About 27 percent or 4,300 
of the nation's districts had effective schools programs in approximately 
27,000 schools; these schools used (1) school teams on which teachers 
and administrators work together to plan and monitor their programs 
and (2) written plans for improving school effectiveness. Program 
experts recommend school teams and written plans to help foster 
school-based planning and decisionmaking. 

When adding the criteria that districts also evaluate their effective 
schools programs by disaggregating (that is, analyzing separately) aca- 
demic achievement data by student socioeconomic status or ethnicity, 
about 13 percent or 2,100 of the nation's school districts had such pro- 
grams in approximately 18,000 elementary and secondary schools. Pro- 
gram experts consider this evaluation method particularly important 
because it permits schools and districts to specifically identify academic 
achievement gains made by low-income and minority children. Without 
disaggregating achievement data in this manner, a district could incor- 
rectly interpret districtwide achievement gains as improvement among 



Many School Districts 
Operate Effective 
Schools Programs 
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all Children, when, in fact, those gains were made by only some children, 
such as those from higher-income families. 



Which Schools Have 
Effective Schools 
Programs? 



Effective schools programs operate in elementary and secondary schools 
at an average rate approximately commensurate with districts' average 
total percentage of elementary and secondary schools. For example: An 
average of about 53 percent of effective schools programs operate in 
elementary schools; similarly, elementary schools make up, on average, 
52 percent of the total number of schools in the districts with programs. 
An average of about 20 percent of effective schools programs operate in 
high schools; similarly, high schools make up, on average, 21 percent of 
the total number of schools in the districts with effective schools pro- 
grams. In addition, these programs commonly serve mixed enrollments 
of low-income and higher-income children. 



Which Districts Have 
Effective Schools 
Programs? 



Effective schools programs are somewhat more likely to operate in large 
school districts, but are about as common in urban and nonurban areas. 
For example, although districts with enrollments of 10,000 students and 
over make up 4 percent of all U.S. districts, they account for about 8 
percent of the districts with effective schools programs. These programs 
operate in urban school districts, including central city districts, at a 
rate nearly commensurate with the percentage of such districts in the 
nation. 



Many Districts Require questionnaire results indicate that ill half of all school districts with 

Schools to Implement effective schools programs, some or all schools with the programs were 

Effective Schools -equired by their districts to implement them. la the other half of the 

Prntyrs^mc districts with programs, schools voluntarily implemented them. 



Programs Frequently 
Developed and 
Implemented With 
External Assistance 



School districts frequently receive external assistance m developmg and 
implementing effective schools programs from state education ager-'ies 
(SEAS), universities, federally funded regional educational laboratories, 
and other agencies or individuals outside school districts. Our question- 
naire results indicate that about 64 percent of the districts with effec- 
tive schools programs received such external assistance, most often 
from SEAS. We identified 31 seas that assisted districts and schools in 
developing or implementing effective schools programs. 
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School districts reported using a variety of student performance meas- 
ures, ranging from achievement tests to student dropout rates, to evalu- 
ate their programs' effect on academic achievement. When conducting 
these evaluations, however, school districts seldom determine whether 
the academic achievement of low-income or minority children is 
improved; this is because districts seldom disaggregate academic 
achievement data by student socioeconomic status or ethnicity. 

Approximately 83 percent of all school districts with effective schools 
programs evaluate their programs using achievement tests, which typi- 
cally measure basic language arts and mathematics skill levels or assess 
mastery of specific curriculum objectives. About 75 percent of these dis- 
tricts use nontest measures, which commonly include grades, dropout 
rates, attendance, and enrollment rates in advanced and remedial 
classes. Sixty-six percent of the districts with programs use both test 
and nontest measures. 

Only about 12 percent of the districts with effective schools programs 
regularly disaggregate achievement test results by student socioeco- 
nomic status. An even smaller percentage (about 9 percent) do so by 
ethnicity. Approximately 22 percent of the districts disaggregate 
nontest measures by student socioeconomic status, and about 29 percent 
do so by ethnicity. 



As requested by your office, we did not obtain official agency comments 
on this report. We did, however, discuss its contents with Department of 
Education officials and have incorporated their comments where appro- 
priate. As arranged with your office, unless you publicly announce its 
contents earlier, we plan no further distribution of this report for 7 days 
from its issue date. At that time, copies will be sent to the Secretary of 
Education and other interested parties. The m^or contributors to this 
report are listed in appendix VIII. 

Sincerely yours. 



William J. Gainer 

Director, Education and Employment Issues 
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Effective Schools Programs: Their Extent 
and Characteristics 



Background 



The Hawkms-Stafford Elementary and Secondary School Improvement 
Amendments of 1988 (P.L. 100-297) authorized Chapter 1 compensatory 
education funds and Chapter 2 educational improvement block grant 
funds to bfe expended for effective schools programs (esps). An effective 
schools program is a school-based program to improve the academic 
achievement of all children in a school, regardless of socioeconomic 
backgrounv-il or ethnicity, and in particular educationally deprived chil- 
dren (see fig. 1). These programs are also designed to promote school- 
level planning and decisioiunaking and to become formal processes by 
which schools assess their needs and plan for their own improvement 
Effective schools programs, like other school improvement efforts such 
as parental choice in selecting schools for their children to attend and 
increased graduation requirements, have become increasingly common 
as the educational reform movement has burgeoned during the 1980s. 



Figure 1 




Effective Schools Programs 

are school-based programs 
to improve the academic 
achievement of all children, 
regardless of socioeconomic 
status or ethnicity, and 
particularly educationally 
deprived children. 
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and Characteristics 



Figure 2 



GAO How Schools Typically 
Implement ESPs 



• Acquaint school staffs with 
research findings 

• Establish teams of teachers 
and administrators 

• Assess schoolwide and 
classroom needs 



Formulate improvement plans 



Effective schools programs have been developed by individual schools, 
school districts, state education agencies (SEAs), federally funded 
regional educational laboratories, and universities. According to the 
]-esearch literature and experfc:, we interviewed, effective schools pro- 
grams differ in (1) the amount of technical and financial assiscance 
received from sources outside the school, (2) the amount of training 
given staff, (3) the emphasis on schoolwide ve*-<^'^ clabi>room character- 
istics, and (4) the variety and use of school data tuat are obtained for 
evaluation purposes. Despite these differences, the literature shows that 
schools typically implement effective schools programs in a similar man- 
ner (see fig. 2). 
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Effective Schools Programs: Their Extent 
and Characteristico 



Effective Schools Research Effective schools programs seek to develop or improve on school charac- 
teristics identified by effective schools research as associated with high 
student achievement (see app. 1 for a list of 22 such characteristics). 
This research includes studies on school effectiveness, teacher effective- 
ness, organizational management, and program implementation, accord- 
ing to the literature we reviewed. 

Most typically, effective schools programs emphasize the findings of 
studies on school effectiveness.* In these studies, researchers have iden- 
tified characteristics present in schools with unexpectedly high stan- 
dardized student achievement test scores given their students' 
socioeconomic background. Most of these studies focused on urban ele- 
mentary schools serving low-income and minority students. These stud- 
ies challenged the assumption that schools could have little effect on 
improving student achievement — a popular interpretation of earlier 
studies on equality in education, which found a direct relationship 
between socioeconomic background and student achievement.^ 

The findings of school effectiveness research were rapidly adopted by 
schools, districts, and states as models for school improvement pro- 
grams during the 1980s. The growth of effective schools programs was 
encouraged, in part, by (1) the belief among educators that schools could 
make a difference in improving student achievement and (2) school 
effectiveness researchers who advocated programs based on their 
research. The most influential researcher and advocate of these pro- 
grams was the late Ronald Edmonds, who, as a New York City school 
administrator, launched that city's effective schools program in 1978. 



See, for example, W.B. Brookover and L.W. I^zotte, Changes in School Characteristics Coincident 
with Changes in Student Achievement (East Lansing: institute for Research on Teaching, Michigan 
State University, 1979); M.M. Rutter and others, Fifteen-Thousand Hours: Secondary Schools and 
their Effects on Children (Cambridge, Mass.: Harvard University Press, 1979); 6. Weber, Inner^fcity 
Children Can Be Taught to Read: Four Successful Schools (Washington. D.C.. Counal for Basic Educa- 
tion, 1971). 

^Jarnes Coleman and others. Equality of Educational Opportunity (Washington. D.C.: National Center 
for Education Statistics, 1966); Christopher Jenks and others. Inequality; A Reassessment o f the 
Effect of Family and Schooling in America (New York; Basic Books, 1972). 
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Effective Schools Programs: Their Extent 
and Cliaracteristics 



Figure 3 



Characteristics Frequently 
Cited in Effective Schools 

• Strong principal leadership 

• A pervasive and broadly 
understood instructional focus 

• Safe & orderly school climate 

• High teacher expectations for 
student achievement 

• Student achievement data used 
for evaluating program success 



Edmonds defined an effective school as one in which equal percentages 
of students from varying socioeconomic backgrounds achieve amini- 
muiti level of mastery in basic skills. He selected five characteristics 
identified by school effectiveness research and popularized their use as 
a model for school improvement (see fig. 3). 

Literature describing existing effective schools programs indicates that 
they make extensive use of this five-characteristic model, although 
many variations exist. In addition, these five charactenstic^ are incorpo- 
rated in the definition of an effective schools program used in the Haw- 
kins-Stafford Amendments of 1988 (see app. 2 for the list in P.L. 100- 
297). 
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and Characteristics 



In some of the literature we reviewed, researchers were critical of the 
\yidespread growth of programs based on the findings of school effec- 
tiveness research .3 Criticism of the research is leveled against the (1) 
reduction of the findings to a set of specific characteristics, Including 
the five just mentioned, to be used as a formula for school improvement; 
(2) generalization of the findings of studies conducted In urban elemen- ' 
tary schools to all levels of schools in urban and nonurban areas; (3) 
notion that once aware of a set of specific characteristics, schools can 
simply decide to adopt them; and (4) lack of causal evidence about what 
actions might bring about these specific characteristics in a school. 



Analyzing Academic when schools and districts analyze students' academic achievement to 

Achievement to Evaluate evaluate the results of effective schools programs, uoth the experts we 
Effective Schools interviewed and the literature we reviewed indicated that measures of 

Programs academic achievement should be disaggregated (that is, analyzed sepa- 

rately) by student socioeconomic status and ethnicity.-* Distinguishing 
the academic achievement of low-income and minority students permits 
schools and districts to determine whether children in these subgroups 
show improvement. Without separate analyses, achievement data would 
remain aggregated, and a school or district could incorrectly interpret 
achievement gains among only some children— for example, those who 
are nonminority or from higher-income families— as improvement 
among all children. 



How school enrollments are mixed demographically may influence 
which student subgroups are relevant for disaggregation, according to 
some experts we interviewed. For example, in districts where schools 
with the programs by and large serve only children from low-income 
families, it may not be relevant to disaggregate by student socioeco- 
nomic status. 



S^' ^^^P^*^- Rowan, S. Bosscrt, D. Dwyer, "Research on Ef fccU ve Schools: A Cautionary 
^pfL.^"f u Researcher. Vol 12, No.4 (April 1983), pp.24^1; S.C. Purkey and M.S. Smith. 
Effective Schools: A Review," Th< Elementary SchoolJouma' Vol. 83, No. 4 (1983), pp 427-51: M 
Cohen. Instructional Managemen. and Social Conditions in Effective Schools," in School Finance and 
School Improvement: Linkages ir. the 1980s. A. Odden and L.D. Webb, eos. (Washi ngton, B.C.. Amt-ri- 

Educauonai Mnance Association, 1983); J. D'Amico, "Each Effective School May Be One 'of A 
Kind. • Educauonai Leadership (Dec. 1982), pp.61-62. 

••student socioeconomic status can be approximated with actual family income, level of parent educa- 
tion, student partiapation in the free or reduced-price lunch (or other public assistance) program, and 
the language spoken at home. /i- 
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nnd Characteristics 



Figure 4 



GAD Federal Funds Authorized 
for ESPs 



ESPs may be funded with 

• Chapter 1 funds for 
schoolwide projects 

• Chapter 2 funds for local 
and state programs 



Some schools and districts with effective schools programs seek to nar- 
row the performance gap between students; thus, they may establish 
goais for raising the academic achievement of a certain percentage of 
lower-achieving students to a higher level. To detennine whether such a 
goal is met, a school or district could analyze achievement measures for 
students grouped according to their prior achievement level, without 
disaggregating the data by student socioeconomic status or ethnicity. 
Th?s method of evaluating program results, however, would not disclose 
whether subgroups of children (for example, low-income or minority) 
that were lower achievers in an earlier period show improved perform- 
ance in a later period. 
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Federal Support for 
Effective Schools 
Programs 



Public Law 100-297 includes the first Chapter 1 and Chapter 2 program 
authorizations for effective schools programs (see fig. 4). In fiscal year 
1989, a total of $4.6 billion was appropriated for Chapter 1 and $462.8 
million for the Chapter 2 block grant program. 

Chapter 1 funds may be used for effective schools programs when dis- 
tricts fund these programs as schoohvide projects. Such projects may 
only operate in schools where at least 75 percent of the children enrolled 
are from low-income families. The purpose of schoolwide projects is to 
upgrade a school's entire educational program rather than serve only 
certain students like other Chapter 1 programs. Chapter 2 funds may be 
used by distncts for effective schools programs as innovative programs 
to carry out schoolwide improvements— one of six authorized local 
targeted assistance programs (see app. 3). Public Law 100-297 requires 
SEAS to use a portion of the Chapter 2 funds they retain for effective 
schools programs (see fig. 5). 

SEAS must distribute at least 80 percent of their Chapter 2 funds to 
school districts for use among the authorized targeted assistance pro- 
grams. The other 20 percent is reserved for state programs that include 
(1) Chapter 2 program administration, (2) assistance to school districts 
and statewide activities to provide targeted assistance, and (3) assis- 
tance to school districts and statewide activities to support effective 
schools programs. 

Of Chapter 2 funds reserved for state programs, seas are required to use 
at least 20 percent for effective schools programs, unless a waiver is 
granted by the Secretary of Education. The Secretary may waive this 
requirement if a state already spends twice the required amount from 
nonfederal sources on these programs. The total amount of state 
Chapter 2 funds available for effective schools programs in fiscal year 
1989 was about $18.4 million. Individual state funds available for effec- 
tive schools programs ranged from about $92,000 in states such as Dela- 
ware and Vermont to $1.9 million in California. 
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RgureS 



GAO State Distribution of 
Chapter 2 Funds 



SEAs distribute at least 80% 
of funds to districts -- 

•To be used for six local 
targeted assistance 
programs, including ESPs 

Remaining 20% of funds 
reserved for state use 
•States must use a portion 
for ESPs 
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Effective Schools Programs: Their Extent 
and Characteristics 



Figure 6 



GAO Study Objectives 


Determine 


Number of U.S, districts with 
eTTeciive scnoois programs 


Identify 
Describe 


Common ESP characteristics 
and practices 

How districts evaluate program 
effect on achievement 


Discuss 


Federal requirements for 
evaluating ESPs 
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and Characteristics 



Figure 7 



GAO Methodology 



Interview Federal and state officials 

Researchers / program experts 

Review Effective schools research 



Survey A nationally representative 
sample of school district 
superintendents 



Objectives Scope and ^° provide needed information on trie extent and characteristics of 
MptVinHnlncyv ' effective schools programs in the nation's school districts, the Chairman 

ivietllUUUlUgy of the House Education and Labor Committee asked us to conduct a 

national study of these programs. In response to his request, we defined 
our objectives as shown in figure 6. The methodology used to accomplish 
our objectives is summarized in figure 7. To help us plan our study and 
develop a questionnaire for 1,685 school district superintendents, we (1) 
interviewed experts as well as Department of Education and sea offi- 
cials and (2) reviewed the research literature on effective schools. The 
questionnaire, which was the primary means through which we 
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obtained information about effective schools programs nationwide, cov- 
ered topics relating to the development, implementation, and evaluation 
of programs operating in school year 1987-88. 

We mailed copies of the questionnaire to the superintendents of a strati- 
fied random sample of school districts across the country; we received 
responses from 82 percent of the school districts in our sample. Our 
findings are representative of the approximately 16,000 school districts 
in the nation.^- ^ In cases where information was incomplete or discrep- 
ancies occurred in districts' responses, we spoke with the respondents 
and made appropriate changes. We did not visit districts to verify the 
accuracy of their responses. 

We used a stratified sample design to obtain national representativeness 
and minimize sample size. The strata for our sample consisted of (1) 310 
school districts, selected randomly from 873 districts identified by 
experts as having effective schools programs— including school-based 
planning teams — in which student achievement was disaggregated by 
student socioeconomic status, (fl) 223 school districts, selected randomly 
from 309 districts identified by experts as having effective schools pro- 
grams — including school teams — but with unknown evaluation prac- 
tices; (3) the 50 largest school districts of the nation's 50 largest cities 
(see app. V for a list of cities);^ and (4) 1,089 school districts, selected 
randomly from the 1986-87 public education agency universe of the 
Department of Education's common core of data (ccd).^ 

We discussed our questionnaire results with Department of Education 
officials and incorporated their comments where appropriate. We con- 
ducted our study during October 1988 through April 1989 in accordance 
with generally accepted government auditing standards. 



Our sample only includes local school districts, which do not include regional education service agen- 
cies, supervisory union administrative centers, state-operated agencies, and federally operated agen- 
cies, which together total approximately !,300 districts. (Source: VS. Department of Education, 
Center for Education Statistics.) 

^The confidence intervals for our data are shown in appendix IV. 

^Wa selected the fifty largest cities based on 1986 Bureau of the Census population estimates. 

®CCD is a universe survey of U.S. schools and districts and is not subject to sampling error. It consists 
of data suitted annually by all SEAs in response to six survey instruments administered by the Center 
for Education Statistics, which maintains the database. The 1986-87 CCD public education agency 
universe had a response rate of 91 percent 
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Figure 8 



Criteria Used to 
Measure Extent of ESPs 



Criterion 1 



Scliool improvement programs 
based on findings of effective 
schools research 



Criterion 2 Also use school teams and 

written improvement plans 



Criterion 3 Also evaluate ESPs by 

disaggregating achievement 
data by student socioeconomic 
status or ethnicity 



Mar- 



ly School Districts 
Operate Effective 
Schools Programs 



We used various criteria to measure the extent of effective schools pro- 
grams to provide the Congress with information on (1) the overall 
number of districts and schools With the programs and (2) the extent 
that these programs include specific activities recommended by program 
experts (see fig. 8). 
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Figure 9 



GAO Extent of ESPs in U.S. Districts 
Meeting Criterion 1 



Majority have or plan 
to start programs 

Not Planning to implement an ESP 




Had ESP in School Year 1987-88 



Planning to Implement an ESP in School 
Years 1988-89 or 1989-90 



Using the first criterion, which stipulates that the programs be based on 
the findings of effective schools research, about 41 percent or 6,500 of 
the nation's school districts had effective schools programs in operation 
in approximately 38,000 elementary and secondary schools during 
school year 1987-88 (see fig. 9). An additional 17 percent or about 2,600 
of the nation's districts have plans to implement effective schools pro- 
grams during school years 1988-89 or 1989-90. Many of the programs 
have been recently established (see fig. 10). Over half of the district offi- 
cials reported that their effective schools programs were first imple- 
mented during school years 1986-87 or 1987-88. 
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Figure 10 



GAO 



Many ESPs Recently 
Implemented 



Over half began during 
school years 1986-88 



Pwcmt o» School DIttricts WRh ESPi HMlIng Crtlirionl 




S«fora 1»90« lM2'e« ieM-«S 18««48 
School Vun ESPa Pint Iiiipl«iK.in«: 



The second program criterion includes those programs that use (1) 
school-level teams on which teachers and administrators work together 
to plan and monitor their programs and (2) written plans for improving 
school effectiveness; about 27 percent or 4,300 of the nation's districts 
had such effective schools programi, in approximately 27,000 schools 
(see fig. 11 and see app. VII). Program experts encourage school teams 
and written plans to help foster school-Jev( planning and decisionmak- 
ing by school staff, which (as discussed on p. 10) are objectives of effec- 
tive schools programs. 
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Figure 11 



GAO Using Different Criteria, 

Extent of ESPs in U.S. Districts 



45 P#rc«ntol School OUtrtctsWHhESPi 
40 
35 
30 
25 



10 



/ 



Crll«ii« Uttd to Uttturt Exttnt ot ESPi 



When using the third criterion— that districts also disaggregate aca- 
demic achievement data by student socioeconomic status or ethnicity 
when evaluating their programs— only about 13 percent or 2,100 of the 
nation's school districts had effective schools programs in approxi- 
mately 18,000 elementary and secondary schools (see fig. 11)." As dis- 
cussed on pages 14 and 15, such analyses permit schools or districts to 
determine whether low-income and minority children show 
improvement. 



¥or the purpose of criterion 3, academic achievement data may be obtained using achievement tests 
or nontcst measures such as student grades, attendance, and dropout rates. 
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Figure 12 



GAO Which Schools and Districts 
Have ESPs? (Criterion 1 ) 



• ESPs are about as likely in 
schools at each grade level 

• ESPs commonly serve mix of 
low-income and higher-income 
children 

" ESPs are more likely in large 
districts 

• ESPs are about as likely in 
urban and nonurban districts 



Which Schools and describe which schools and districts have effective schools programs, 

nicinVf o Wt^Tr^ collected data on the grade levels of schools with the programs, the 

uibtl ndVe socioeconomic diversity and size of their enrollments, and their geo- 

Ef f eCtive Schools graphic location (see fig 1 2). 

Programs? 



Programs About as Likely Effective schools programs meeting the first criterion operated m cle- 
at Each Grade Level mentary and secondary- schools at an average rate approximately com- 
mensurate with districts' average total percentage of elementary and 
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Figure 13 



GAD ESPs About as Likely 
at Each Grade Level 



P«rc«nt of Schoote In Districts WHh ESPs 
55 




Grmda Lsvil of Schoote fn Districts With ESPs 



I S€txx>lsWiihESPs 
AO Schools 



secondary schools, district officials reported (see fig. 13). For example, 
an average of about 53 percent of effective schools programs operate in 
elementary schools, with elementary schools making up, on average, 52 
percent of the total number of schools in districts with programs. In 
addition, an average of 20 percent of effective schools programs operate 
in high schools, with high schools making up, on average, about 21 per- 
cent of the schools in districts with programs. 
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Figure 14 



Socioeconomic Status of 
Children 



ESPs serve mf^^ of low-income 
and higher-income 



Percent of School 
Districts WUh ESPs 
20 




0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 

Percent of Children on froo or Reducod*Prfca Lunch In Schools With ESPs 



71-80 81-60 91-100 



Prograins Commonly Serve 
Mix of L<ow-Income and 
Higher-Income Children 



Effective schools programs meeting the first criterion commonly serve 
mixed enrollments of low-income and higher-income children, district 
officials reported (see fig. 14). To estimate the socioecononiic diversity 
of districts' school enrollments, we asked district administrators to spec- 
ify the percentage of children who participated in the free or reduced- 
price lunch program in those schools with effective schools programs.'^ 



The percentage of children participating in the free or reduced-price lunch program may underesti- 
mate the percentage of low income children in a school because all eligible children do not participate. 
Children that receive free lunch have family incomes of 130 percent or less of the official poverty 
threshold, which was $9,431 for a 3-person family in 1988, children that receive reduced-price lunch 
have family incomes of 185 percent or less of the official poverty threshold. Among all children that 
received free or reduced-price lunches during the first 6 months of fiscal year 1989, 85.5 percent 
received free lunches. (Source. U.S. Department of Commerce, Bureau of the Census and U.S. Depart- 
ment of Agriculture, Child Nutrition Division.) 
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FiGure 15 



GAD 



Large Districts More Likely to 
Have ESPs Than Small Districts 



Percsnt of School Districts 
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StudGnt EnroIImoiTt 



2,500-9,939 



School Districts With ESPs 
All U^. School Dtstrtcts 




1-2.499 



Programs More Likely in Effective schools programs are somewhat more likely to operate in large 
Larger School Districts ^^^^^^ districts (em-ollments of 2,500 or more). For example, although 

districts with enrollments of 10,000 students and over make up 4 per- 
cent of all U.S. districts, these districts account for about 8 percent of 
the districts with effective schools programs meeting the first criterion 
(see fig. 15). Likewise, districts with enrollments of 2,500 to 9,999 stu- 
dents make up 18 percent of all U.S. districts and about 29 percent of 
the districts with programs. 
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Figure 16 



GAD ESPs About as Likely in Urban 
and Nonurban School Districts 



60 Percent of School Districts 
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Primarily PAS A b«t Net Outsjdfi of MSA 

Certral Cftyof Primarily 
MSA Central Cfty 

Geographical Area Served by School Districts 



School Districts With ESPs 
AQ U^. School Districts 



Programs About as Likely 
in Urban and Nonurban 
Areas 



Effective schools programs operate in urban school districts, including 
central city districts, at a rate nearly commensurate with the percentage 
of such districts in the nation. We defined urban districts as districts 
that serve a metropolitan statistical area (msa) and central city districts 
as districts that serve the central city of an msa." For example, central 
city school districts make up 4 percent of all U.S. districts and about 
5 percent of the districts with effective schools programs meeting the 



An MSA has one or more central counties containing an urbanized area with at least 50,000 inhabi- 
tants. An MSA may also include outlying counties that have close economic and social relationships 
with the centra] counties. 
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Figure 17 



GM) School Characteristics 
Emphasized by ESPs 



Many districts cited ali nine 
of these characteristics 

« Strong instt uctional leadership 

• High achievement expectations 

• Pervasive instructional focus 

• Monitoring student achievement 

• Basic skills acquisition 

• Staff collaboration 
Academic learning time 

• Parental support " 

• Safe & orderly school climate 



first criterion (see fig. 16). We found that at least 39 of the nation's 50 
largest cities had effective schools programs in their districts during 
school year 1987-88 (see app. V). 



Effective schools programs seek to develop or i nr. prove on school charac- 
teristics identified by effective schools research as associated with high 
student achievement (see p. 12). We asked district administrators to 
indicate which school characteristics their programs emphasized 
through specific activities such as needs assessments, staff development 



School Characteristics 
Emphasized by 
Programs 




31 

^^^^ GAO/HRD.89.132BR Effective Schools Programs 



Effective Schools Programs; Their Extent 
and Characteristics 



training, and changes in classroom instruction and school management 
procedures J2 According to their responses, in about 88 percent of the 
6,500 districts, the programs sought to improve on at least six of nine 
school characteristics listed in our questionnaire (see fig. 17). The pro- 
grams in about 67 percent of the districts sought to improve on at least 
eight characteristics, and the programs in about 53 percent of the dis- 
tricts emphasized all nine (see app. VII for further details on fig. 17). 

A study that used a nationally representative survey of high schools 
provides another example of effective schools programs emphasizing 
many different characteristics. »3 In this study, a majority of principals 
reported that effective schools programs in their schools gave major 
emphasis to at least 12 characteristics identified by effective schools 
research. The authors of this study questioned how the schools could 
devote major emphasis to so many characteristics simultaneously since 
each one was likely to require considerable effort. 



*20ur questionnaire listed nine school characteristics that were frequently cited in the literature on 
effective schools research (see fig. 17). 

^^S.C Purkey, R.A. Rutter, and F.M. Newmann, "VS. High School Improvement Programs: A Profile 
from the High School and Beyond Supplemental Sur\'ey " Metropolitan Education . Vol.3 (Winter 
1986-87), pp. 59-91. The survey used was the 1984 High School and Beyond Supplemental Survey of 
Principals and Teachers. 



Page 31 ^2 GAO/HRD-89-132BR EffecUve Schools Programs 



Effective Schools Programs: Their Extent 
and Characteristics 



Figure 18 



GAO 



School District 
Involvement in ESPs 



Half of the districts required 
schools to implemekit programs 



Districts in Which All Schools Wrth ESPs 
Voluntarily Chose to Have Them 




Districts in Which All Schools With ESPs 
Were Required to Have Them 



4% 

Districts in Which Some Schools With 
ESPs Were Required to Havo Them 



Many Districts 
Require Schools to 
Implement Effective 
Schools Programs 



Our questionnaire results indicate that in half of the school districts 
with effective schools programs meeting the first criterion, some or all 
schools with the programs were required by their districts to implement 
them (see fig. 18). In the other half of the districts with programs, 
schools implemented them on a voluntary basis. Some districts that 
require effective schools programs do so because of state laws that 
require (or encourage) school improvement programs (see fig. 19). 
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Rgure 19 



GM) Examples of State Laws that 
Pertain to ESPs 



States such as Maine, Indiana 
New York, and South Carolina , 

• Require or fund improvement 
programs in scliools, or 

• Require schools to prepare 
improvement plans, or 

• Have accreditation standards 
that include effective schools 
objectives 



Some of the literature we reviewed discussed t\e strengths and weak- 
nesses of approaches in which districts become involved in schools' deci- 
sions to implement an effective schools program.^^ 

• Top-down approach : The school district requires schools to implement 
effective schools programs and determines program content, goals, and 
expected outcomes, usually focused on achievement tests. A strength 

^^See, for example, S.C. Purkey and M.S. Smith, "School Reform. The District Policy Implications of 
the Effective Schools Literature," The Elementary School Journal. Vol. 45, No. 3 (Jan. 1985), pp. 353- 
63; L. Cuban, "Transforming the Frog into a Prince Effective Schools Research, Policy, and Practice 
at the District Level," Harvard Educational Review. Vol. 54, No. 2 (May 1984), pp. 129-37. 
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cited in this approach is that reform will be attempted where it is 
needed. But a weakness may be that the lack of staff choice in program 
implementation and management, as well as the focus on test scores, 
may diminish (1) school staffs' sense of responsibility and commitment 
toward the program and (2) flexibility in curricula, classroom practices, 
and program management. 

• Bottom-up approach : The school district may provide incentives, such as 
program funds and teacher-release time from classes for program plan- 
ning, but each school voluntarily chooses to implement an effective 
schools program, determines its own agenda, and uses district funds in 
the manner that staff and parents choose. Choice and flexibility are 
strengths of this approach, but a weakness cited is that schools that 
could benefit from the program may choose not to participate or may 
drop out as staff enthusiasm declines. 

• Mixed approaches : This approach combines the strengths of the other 
two approaches. The district may require schools to implement effective 
schools programs and provide incentives that encourage staff planning 
and collaboration, such as funds ^or substitutes, but then relies on 
school staffs for program design, implementation, and management. 



Programs Frequently 
Developed and 
Implemented With 
External Assistance 



SEAS, universities, federally funded regional educational laboratories, 
and other agencies or individuals outside of the school district fre- 
quently assist districts and schools in developing and implementing 
effective schools programs, according to our survey results and the liter- 
ature we reviewed. To assist principals and teachers plan and implement 
their programs, agencies and individuals outside of the school district 
may develop program models and provide staff training and technical 
assistance; seas may provide financial support to districts and schools. 
Our questionnaire results indicate that about 64 percent of the districts 
with effective schools programs meeting the first criterion received 
external assistance in developing their programs. Many of these pro- 
grams were developed with assistance from an sea (see fig. 20). 
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Figure 20 
I 



GAO Sources of External 
Assistance for ESPs 



Many districts received 
assistance from SEAs 

Porcont of Schod Districts That Rocslvad ExtsnwU Asslsttnco 
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Figure 21 



GAD Examples of SEA 
Support for ESPs 



• Connecticut and Minnesota 
SEAs have ESP model 

• New York, Michigan, and 
Kentucky SEAs provide training 
and technical assistance 

• California, New Jersey, 

and Wisconsin SEAs provide 
financial support to schools 
or districts 



Through interviews with state officiais and program experts in all 50 
states and the District of Columbia, we identified 31 seas that assisted 
schools and districts in developing or implementing effective schools 
programs (see fig. 21 and see app. VI for a listing of seas and examples 
of the assistance they provide). We found that sea staff provided assis- 
tance through such means as school visits and staff development work- 
shops, including a train-the-trainers approach, in which sea staff train 
district staff to train key school staff. For example, an assistant superin- 
tendent trained school principals in Marion School District 1, South Car- 
olina (see fig. 22). A trainer from sea observed and evaluated this 
workshop. 
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Figure 22; Train-The-Tralners Staff Development Training 
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Figure 23 



^ How Do School Districts 
Evaluate ESPs? 



• Districts use achievement 
tests and nontest measures, 
but... 

• Districts seldom 
disaggregate achievement 
data by student socioeconomic 
status.or ethnicity; thus, 

they seldom determine if 
ESPs help all children 



School district administrators reported using various achievement tests 
and nontest measures to evaluate the results of effective schools pro- 
grams on student academic achievement (see fig. 23). When evaluating 
their programs, however, school districts seldom determine if the aca- 
demic achievement of low-income and minority children is improved 
because the districts seldom disaggregate achievement data by student 
socioeconomic status or ethnicity. 



Districts Seldom 
Determine If Programs 
Help AU Children 
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GAO Program Evaluation 



Districts use achievement 
tests and nontest measures 



9% 

Usod Nontest Moasuros Only 
8% 

Used No Achkivomont Tosts or Nontost 
Measures 




Usod Both Achievement Tests and 
Nontost Measures 



Used Achievement Tests Only 



Various Measures of 
Academic Achievement 
Used 



The m^ority of school districts use both achievement te;*ts and nontest 
measures to evaluate their programs (see fig. 24). The achievement tests 
ased by school districts typically measure basic language arts and math- 
ematics skill levels or assess mastery of specific ' ^rriculuiri objectives. 
Nontest measures used by school districts may include studer.t grades, 
attendance, dropout rates, and enrollments in advanced and remedial 
classes. 
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Figure 25 



GAO District Evaluations Using 
Acliievement Tests 



Data seldom disaggregated 

Porc«m of Schoc4 Dtetrict* Wtth ESPs 




/// 



No Dsaggregaied An^ys«s Condtned 
Disawresaicd AnaVses Condueiod Oceassionally 
Disapsfesacd Analyses CocxJueied Rejuiarfy 



Academic Achievement of 
Student Subgroups Seldom 
Analyzed 



When evaluating their programs, districts seldom disaggregate achieve- 
ment test results or nontest measures by student socioeconomic status, 
ethnicity, or sex (see figs. 25 and 26). But districts frequently disaggre- 
gate these measures by students' prior achievement level. Analyzing 
achievement measures by prior achievement level may allow a school or 
district to determine whether an increased percentage of students is per- 
forming at a higher achievement level (see p. 15). Conducting such anal- 
yses, however, without disaggregating data by student socioeconomic 
status or ethnicity may fail to disclose whether relevant subgroups of 
children show improved performance among those studento that were 
lower achievers in an earlier period. 
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Figure 26 



^ District Evaluations Using 
Nontest Measures 




Data seldom disaggregated 
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Rgur8 27 



GAD In Districts Not Disaggregating 
By Socioeconomic Status 

ESPs serve mix of iow-income 
and higher-income children 



25 Pftfcsnt of Districts That Do Not Disaggregato Data By Sodoeconondc Status 




Percent of ChUdren on Froo or Roduccd-Prico Lunch In Schools With ESPs 



Since the demographic mix of school enrollments may influence which 
student subgroups are relevant for separate analyses (see p. 14), we 
examined the socioeconomic diversity of the children sei-ved oy the pro- 
grams in those districts that did not disaggregate achievement measures 
by student socioeconomic status. In such districts (see fig. 27), effective 
schools programs commonly served mixed enrollments of low-income 
and higher-income children. Because these districts do not distinguish 
the academic achievement of their low-income children, the districts 
could incorrectly interpret achievement gains among only some previ- 
ously lower-achieving children — for example, those from higher-income 
families — for improvement among all children. 
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^ Chapter 1 and 2 Evaluation 
Requirements in P.L. 100-297 

The Department must conduct 
a national study of ESPs 
funded under Chapter 2 

► States must evaluate Chapter 
1 and 2 programs and submit 
evaluations to the Department 

The Department must 
summarize these state 
evaluations and periodically 
report to Congress 



Federal Evaluation ^^^^^ 100-297 requires the Department of Education to contract to 

Rpm 11 rpmpntc fnv conduct a national study on effective schools programs funded under 

SS^ r. ^"r* , ^^^P^^^ 2 (see fig. 28). The law also requires seas to evaluate programs 

JCilieCtlVe bCnOOiS supported with Chapter 1 and Chapter 2 funds, using data collected 

Programs ^^^^ school districts; the law, however, requires no specific state or 

° local evaluations of effective schools programs. 

SEAS must submit their evaluations to the Department biennially for 
Chapter 1 programs and in fiscal year 1992 for Chapter 2 px-ograms. The 
Department in turn must provide the Congress with evaluations of 
Chapter 1 and 2 programs through reports summarizing sea evaluations 



ERIC 



Page 43 ^ 4 GAO/HRIV8C.;32BR Effective Schools Programs 



Effective Schools Prograins; Their Extent 
/ and Characteristics 



of local programs. The Department must submit these reports biemiially 
under Chapter 1 and by October 1, 1992, under Chapter 2. Public Law 
100-297 requires the Department to develop (1) national sta/idards for 
state and local evaluation and reporting purposes under Chapter 1 and 
(2) a model data collection and reporting system that defines specific 
data categories to be used by states, at their discretion, for required 
state evaluations and reports under Chapter 2. 



Chapter 1 Evaluation and Under Chapter l, effective schools programs may be funded as school- 
Reporting Requirements ^^^^ projects, which may only operate in schools where at least 75 per- 
cent of the children enrolled are from low-income families. Public Law 
100-29T and Chapter I program regulations require districts to annually 
evaluate schoolwide projects operating in their schools and to make the 
results, including achievement and other assessment data, available to 
their SEAs and the public. According to an official in the Department of 
Education's planning and evaluation service, the Department plans to 
examine effective schools programs and their results as part of an 
implementation study of the Chapter 1 program. The study will include 
information on (1) the frequency and nature of schoolwide projects and 
(2) their inclusion of features of effective schools programs. The Depart- 
ment expects the final report of this study to be available by December 
1991.15 

In addition. Public Law 100-297 and Chapter 1 program regulations 
require districts to evaluate their Chapter 1 program as a whole at least 
once every 3 years and to suit the results to their seas, seas, in turn, 
must use this information for state-level evaluations that are suitted to 
the Department, seas must also collect data on the race, age, sex, 
number of children with handicapping conditions, and number of chil- 
dren by grade level served by Chapter 1 programs; seas must annually 
suit these data to the Department. 

Neither Public Law 100-297 nor Chapter 1 program regulations require 
districts or states to collect information on, or evaluate, effective schools 
programs separately from other Chapter 1 programs. Public Law 100- 
297 also does not require the Department to separately evaluate effec- 
tive schools programs funded under Chapter 1. According to the deputy 
director of compensatory education programs, because districts and 
states are not required to operate effective schools programs under 



i^According to the Department, it will also examine effective schools programs and their results in a 
longitudinal evaluation, begimung during school year 1990-91, of the Chapter 1 program. 
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Effective Schools Programs: Their Extent 
and Characteristics 



Chapter 1, the Department believes that federally required data collec- 
tion for these programs would be burdensome to seas and school 
districts. 



Chapter 2 Evaluation and ^^nder Chapter 2, districts may fund effective schools programs as inno- 
Reporting Requirements vative programs to carry out schoolwide improvements— one of six 

authorized Chapter 2 local targeted assistance programs. Because the 
Department of Education has not made final dec sions on the specific 
data categories that the Chapter 2 mod«l data Oj . lection and reporting 
system will include, it is uncertain whether any data specifically on 
effective schools programs will be collected under the state and local 
evaluation provisions of Chapter 2. Data for these programs will be col- 
lected, however, for state and local Chapter 2 reporting purposes, 
according to a draft data collection form we obtained from the 
Department. 

By January 199C, the Department of Education expects to define the 
model system's data categories for evaluation purposes. The Depart- 
ment informed us that it is considering collecting specific data on effec- 
tive schools programs, but wishes to avoid undue burden on the states. 
Proposed data categories for state and local reporting purposes were 
specified during spring 1989. These include (1) the number of children, 
by grade level, served by each authorized purpose for local targeted 
assistance, including effective schools programs, (2) the number of 
schools receiving Chapter 2 funds, and (3) the total amount of Chapter 2 
funds used for each targeted assistance purpose. 

Concerning the required national study of effective schools programs 
funded under Chapter 2, the Congress intends data collection for this 
study to include student achievement outcomes disaggregated by stu- 
dent socioeconomic status and ethnicity.'s The Department informed us 
that it plans to award the contract for the study, if funds are available, 
during fiscal year 1990 and, if so, expects the study to be completed no 
later than fiscal year 1992.'" 



'"U.S. Congre^ House, Elementary and Secondary Education Conference R eport to Accompany H R 
5(Rept 100-567, Apr. la, 1988, p.422). f v i. . 

"The Department said that it also intends to examine effective schools programs and their results in 
an ongoing study of effective instructional practices and curricula in elementary schools with high 
concentrations of disadvantaged students. 
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Appendix I 

Selected Characteristics of Effective Schools 
Cited in Research literature 



Strong instructional leadership 
A safe, orderly, and disciplined school climate 
High expectations for student achievement 
Monitoring student achievement frequently to evaluate progress 
Staff consensus on explicit instructional goals and values 
Collaborative, coHegial instructional planning 
Ongoing, schoolwide staff development training 
Teacher accountability and acceptance of responsibility for student per- 
formance 

A focus on basic-skills acquisition 

An emphasis on higher-order cognitive skills 

Increased academic learning time 

Cooperative activity and group interaction in the classroom 
Teacher responsibility for instructional and classroom management 
decisions 

Clearly recognized principles and guideline's for student behavior and 
performance 

Individual school autonomy and flexibiUty 

Staff stability and continuity 

High levels of parental involvement and support 

District-level support for school improvement 

High teacher morale and sense of community in school 

Schoolwide recognition of academic success 

Teacher empathy, rapport, and personal interaction with students 

Strategies to avoid retaining students in grade 

Source: A compilation of school characteristics that appeared in D.E. 
Mackenzie, "Research for School Improvement: An Appraisal of Some 
Recent Trends," Educational Researcher rVoL12. No.4, Apr. 1983, pp.5- 
17); school characteristics cited in research reviewed in S.C. Purkey and 
M.S. Smith, "Effective Schools: A Review," The Elementary School Jour- 
nal (VoL83, No.4, March 1983, pp.427-452). ~ 
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Appendix 11 

Five School Characteristics in the Definition ^ 
Effective Schools Programs Included in Public 
Law 100-297 



• Strong and effective adrainistrative and instructional leadership that 
creates consensus on instructional goals and organizational capacity for 
instructional problem solving 

• Emphasis on the acquisition of basic and higher-order skills 

• A safe and orderly school environment that allows teachers and pupils 
to focus their energies on academic achievement 

• A climate of expectation that virtually all children can learn under 
appropriate conditions 

• Continuous assessment of students and programs to evaluate the effects 
of instruction 

Source: U.S. House of Representatives, Elementary and Secondary Edu- 
cation Conference Report to Accompan y H.R. 5 (Report 100-567. Anril 
13, 1988, p.85). 



48 




P^^® GAO/HttI>.89-132DR Effective Schools Programs 



Appendix III 

Chapter 2 Block Grant Program: Six Authorized 
Local Targeted Assistance Programs 



Programs to meet the educational needs of students at risk of dropping 
out and students for whom providing an education entails higher-than- 
average costs 

Programs for the acquisition and use of instructional and educational 
materials, including library books, reference materials, computer soft- 
ware and hardware for instructional use, and other curricular materials 
that would be used to improve the quality of instruction 
Innovative programs designed to carry out schoolwide improvements, 
including effective schools programs 

Programs of training and professional development to enhance the 
knowledge and skills of educational personnel, including teachers, 
librarians, school counselors and other pupil services personnel, and 
administrators and school board members 

Programs designed to enhance personal excellence of students and stu- 
dent achievement, including instruction in ethics, performing and crea- 
tive arts, humanities, activities in physical fitness and comprehensive 
health education, and participation in community service projects 
Other innovative projects which would enhance the educational pro- 
gram and climate of the school, including programs fo^ :fted and tal- 
ented students, technology education programs, eariy childhood 
education programs, community education, and programs for youth sui- 
cide prevention 

Source: U.S. House of Representatives, Elementary and Secondary Edu- 
cation Conference Report to Accompany H.R. 5 (Report 100-567, April 
13, 1988, p.83). 
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Appendix IV 



Estimates and Confidence Intervals for Selected 
Questionnaire Responses 





Universe or 




Estimated ranae" 


Variable 


school 
districts 


Estimate 


Lower 
limit 


Upper 
limit 


Number of districts with ESPs meeting 
criterion 1 


15.700 








Had ESP in school year 1987-88 




6.509 


6,053 


6.965 


Planning to implement an ESP in 
school year 1988-89 or 1989-90 




2.597 


2.247 


2.947 


Not planning to implement an ESP 




6.594 


6.132 


7.056 


School years ESPs first implemented 
(percent) 


6.509 








1979-80 and before 




5 


3 


8 


1980-82 




4 


3 


6 


1982-84 




p 


6 


10 


1984-86 




25 


21 


29 


1986-88 




57 


53 


62 


Number of school districts with ESPs by 
program criteria 


Criterion 1 


15.700 


6.509 


6.053 


6.965 


Criterion 2 


6.509 


4.304 


4.022 


4.585 


Criterion 3 


6.509 


2.105 


1.834 


2,377 


Number of schools with ESPs by 
program criteria 


Cnterion 1 


6.472 


33.285 


34.807 


42.087 


Criterion 2 


4.300 


27.431 


24.291 


30.790 


Criterion 3 


2.102 


17.794 


15.103 


20.612 


Use of school teams and written plans 
(percent) 


6.488 








Used school teams 




89 


86 


93 


Used written plans 




71 


67 


75 


Used both 




66 


62 


70 


Grade level of schools with ESPs 
(percent) 


6.339 








Elementary 




53 


50 


56 


Intermediate 




18 


16 


20 


High school 




20 


18 


22 


Other 




10 


7 


12 


Grade level of all schools in districts 
with ESPs (percent) 


6.339 








Elementary 




52 


50 


55 


Intermediate 




18 


16 


19 


High school 




21 


19 


23 


Other 




10 


7 


13 



(continued) 
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Appendix IV 

Estimates and Confidence Intervals for 
Selected Questionnaire Responses 





Universe or 




Estimaled ranqe^ 


VaiiaOie 


school 
districts 


Estimate 


Lower 
limit 


UPDer 
limit 


Percent of children on free or rp'^uced* 
price lunch srrved by ESPs ^Kercenl) 


6.410 








u 




2 


1 


3 


1" lU 




17 


14 


21 






21 


18 


25 






18 


15 


22 


31-40 




17 


14 


21 


41-50 




8 


5 


10 


51-60 




6 


4 


8 


61-70 




4 


2 


6 


71-80 




3 


1 


4 


81-90 




2 


1 


4 


9M00 




1 


0 


2 


Student enrollment In districts with 
ESPs (percent) 


6.452 








10.000 + 




8 


0 


11 


2.500-9.999 




29 


O A 

24 


32 


1 • 2.499 




62 


CO 

58 


67 


Geographic area of school districts with 
ESPs (percent) 


6.452 








Primarily central city of MSA 




5 


3 


7 


MSA but not primarily central city 




41 


37 


46 


Outside of MSA 




54 


50 


59 


Number of school characteristics cited 
by districts (percent) 


6,509 








At least SIX of 9 




88 


85 


91 


At least eiciht nf Q 






63 


72 


All 9 




53 


48 


57 


School characteristics citeJ by districts 
(criteria 1 ESPs) 










Instructional leaucf^jhip 


6.490 


92 


89 


94 


Staff expectations 


6.490 


92 


89 


94 


Instructional focus 


6.490 


90 


87 


93 


Monitoring student achievement 


6.489 


88 


86 


91 


Basic*skills acquisition 


6,474 


89 


86 


91 


Staff collaboration 


6.490 


86 


83 


89 


Academic learning time 


6,490 


83 


80 


87 


Parental support 


6.490 


81 


77 


84 


Safe and orderly climate 


6.490 


76 


72 


80 



(continued) 
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Estimates and Confidence Intervals for 
Selected Questionnaire Responses 





Universe or 




Estimated ranae^ 


Variable 


school 
districts 


Estimate 


Lower 
limit 


Upper 
litr • 


School characteristics cited by districts " " 
(criteria 2 ESPs) (percent) 


instructional leadership 


4.285 


92 


89 


96 


Stafr expectations 


4.285 


92 


89 


95 


Instructional focus 


4.285 


93 


90 


96 


Monitoring student achievement 


4.284 


90 


87 


93 


Basic-skiiis acquisition 






89 


95 


Staff collaboration 


4.285 


89 


85 


92 


Academic learning time 


4.285 


84 


80 


88 


Parental support 


4.285 


85 


81 


89 


•Safe and orderly climate 


4.285 


80 


75 


84 


School characteristics cited by districts 
(criteria 3 ESPs) (percent) 


2.087 








Instructional leadership 








99 


Staff expectations 








98 


Instructional focus 




Q7 


y4 


100 


Monitoring student achievement 




91 


87 


96 


Basic-skills acquisition 




96 


92 


99 


Staff collaboration 




94 


90 


97 


Academic learning time 




89 


84 


94 


Parental support 




92 


88 


96 


Safe and orderly climate 




88 


82 


93 


School district Involvement in ESPs 
(percent) 


6.468 








All schools with ESPs were requirta 
to have them 




46 


42 


51 


Some schools with ESPs were 
required to have them 




4 


2 


6 


All schools with ESPs voluntarily 
chose to have them 




50 


45 


54 


Whether district developed program 
with external assistance (percent) 


6.505 








Yes 




64 


60 


68 


No 




36 


32 


40 


Source of external assistance (percent) 


SEA 


4.161 


64 


58 


69 


University 


4.142 


31 


26 


36 


Educational laboratory 


4,127 


36 


30 


41 


Other 


4,149 


31 


26 


36 



(continued) 
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Estimates and Confldence Intervals ?or 
Selected Questionnaire Responses 





Universe or 




Estimated ranae* 


Variable 


school 
districts 


Estimate 


Lower 
limit 


Upper 
limit 


Measures of achievement districts used 
to evaluate results of ESPs (percent) 


Achievement tests 


6.505 


83 


80 


87 


Nontest measures 


6,503 


75 


70 


78 


Both achievement tests and nontest 
measures 


6,505 


66 


62 


70 


Achievement tests only 


6.505 


17 


14 


21 


Nontest measures only 


6.505 


9 


6 


11 


No achievement tests or nontest 
measures used 


6,505 


8 


6 


11 


Student subgroups by which districts 
disaggregated achievement test 
results (percent) 


Socioeconomic status 


5,386 








Regularly 




12 




15 


Occasionally 




21 


17 


25 


Ethnicity 


5.400 








Regularly 




9 


6 


12 


Occasionally 




18 


14 


21 


Sex 


5.385 








Regularly 




9 


6 


12 


Occasionally 




18 


14 


21 


Prioi achievement level 


5.401 








Regularly 




73 


69 


77 


Occasionally 




20 


16 


24 


Student subgroups by which districts 
disaggregated nontest measures 
(percent) 


4.850 








Socioeconomic status 




22 


18 


26 


Ethnicity 




29 


24 


34 


Sex 




33 


28 


38 


Prior achievement level 




76 


72 


81 



(continued) 
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Appendix IV 

Esdmates and Conflde^vce Intervals for 
Selected Questionnaire ilespon£s;s 





Universe or 




Estimated ranae* 


vanaDio 


school 
districts 


Estimate 


Lower 
limit 


Upper 
limit 


h'ercent oi cnilaren on free or reduced- 
price lunch served by ESPs in 
districts that do not disaggregate 
achievement data by student 
socioeconomic status (percent) 


4.637 








0 




3 


0 


5 


1-10 




20 


16 


25 


11-20 




23 


18 


27 


21-30 
31-40 




19 
14 


15 
10 


24 
18 


41-50 
51-60 




7 
5 


5 
3 


10 
7 


61-70 




3 


1 


5 


71-80 




3 


1 


4 


81-90 




2 


0 


3 


91-100 




b 


c 


C 



true proportion of school districts is belween the ranges specified, 
^ess than one percent. 
^Nol applicable. 
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Effective Sch(X)ls Programs in the Fifty Largest 
aties (School Year 1987-88) 









Or! to ria^ 








Criteria 




CItyb 


1 


2 


3 




-| 


0 


o 

w 


New York, NY 


Yes 


Yes 


Yes 


Austin, TX 


Yes 


ICO 


Ypc 

ICO 


Los Angeles, CA 


Yes 


Yes 


Yes 


Oklahoma City, OK 


Yes 


Mn 


INU 


Chicago, IL 


Yes 


Yes 


Yes 


Kansas City, MO 


Yes 


ICO 


Ypq 

ICO 


Houston, TX 


Yes 


Yes 


Yes 


Fort Worth, TX 


Yes 


Yes 


Yp<5 
ICO 


Philadelphia, PA 


Yes 


Yes 


No 


SL Louis, MO 


Yes 


ICO 


Ypq 

ICO 


Detroit, Ml 


Yes 


Yes 


Yes 


Atlanta. GA 


d 


d 


d 


San Diego, CA 


Yes 


Yes 


Yes 


Long Beach, CA 


Yes 


Yes 


Mo 


Dallas, TX 


Yes 


Yes 


No 


Portland, OR 


Yes 


Yes 


Yp<; 

ICO 


San Antonio, CA 


c 


c 


c 


Pittsburgh, PA 


Yes 


Yes 


Yp<5 
ICO 


Phoenix, AZ 


Yes 


Yes 


Yes 


Miami, FL 


Yes 


ICO 


Yp<5 
ICO 


Baltimore, MD 


Yes 


Yes 


Yes 


Tulsa, OK 


Yes 


ICO 


IXU 


San Francisco, CA 


Yes 


Yes 


Yes 


Honolulu, HI 


Yes 


Yp<? 

ICO 


Ypq 

•CO 


Indianapolis. IN 


Yes 


Yes 


Yes 


Cincinnati OH 


No 




i 


San Jose, CA 


d 


d 


d 


Albuquerque, NM 


Yes 


Yes 


No 


Memphis, TN 


d 


d 


d 


Tucson, AZ 


d 


d 


d 


Washington, D.C 


Yes 


Yes 


Yes 


Oakland, CA 


Yes 


Yes 


No 


Jacksonville, FL 


Yes 


Yes 


Yes 


Minneapolis, MN 


d 


d 


d 


Milwaukee, Wl 


Yes 


Yes 


Yes 


Charlotte. NC 


No 


( 


( 


Boston, MA 


c 


c 


c 


Omaha, NE 


Yes 


Yes 


Yes 


Columbus, OH 


Yes 


No 


No^ 


Toledo, OH 


Yes 


No 


No^ 


New Orleans, LA 


Yes 


Yes 


Yes 


Virginia Beach, VA 


c 


c 


c 


Cleveland, OH 


Yes 


Yes 


Yes 


Buffalo. NY 


Yes 


Yes 


No 


Denver, CO 


Yes 


Yes 


Yes 


Sacramento. CA 


Yes 


Yes 


Yes 


El Paso. TX 


Yes 


Yes 


No 


Newark, NJ 


Yes 


Yes 


No 


Seattle, WA 


d 


d 


d 










Nashville, TN 


Yes 


Yes 


Yes 











^These criteria are shown in figure 10. 



^Cities are presented in order of size. 

^School district intends to implement program during school year 1988-89 or 1989-90. 

response to questionnaire received. 
^District disaggregates academic achievement data by student socioeconomic status or ethnicity 
^Not ap^Jicable 
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Appendix VI 

Examples of SEA Assistance to School Districts 
and Schools for Effective Schools Programs 




California 
Colorado 



Connecticu t 
Deiawa re 
Illinois 



Indiana 



Kentucky 



Louisiana 



Massachusetts 



Maryland 



Michigan 



Minnesota 



Missouri 



New Hampshire 



New Jersey 



Nevada 



New York 



Ohio 
Oregon 



Pennsylvania 



Rhode Island 



South Carolina 



South Dakota 



Texas 



Utah 



Viroinia 



Vermont 



Wisconsin 



West Virginia 



Developed ESP model; provided staff training, technical assistance, and fman cial suooorT 
Technical assistance 



Developed ESP model; provided staff training and technical assistance 
Technical assistance and financia l support 
Technical assistance 



Staff training, technical assistance, and financial support 



Staff training, technical assistance, and financial support 



Financial support 



Financial support 



Developed ESP model; provided staff training and financial support 
Staff training and technical assistance 



Developed HSP model; provided staff training 



Staff training 



Staff training and technical assistance 



Staff training, technical assistance, and financial support 



Developed ESP model; provided stal' training and techn cal assistance 



Staff tf aining and technical assistance 



Staff training, technical assistance, an ^l financial support 
Staff training and financial support 



Technical assistance 



Staff training and technical assistance 



Developed ESP model; provided staff training and technical assistance 



Staff training and technical assistance 



Technical assistance 



Staff training and technical assistance 



Technical assistance and financial sjpport 



Technical assistance 



Financial support 



Staff training and technical assistance 
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Appendix VII 

Tables Supporting Figures in Report Text and 
Supplementary Tables 



Table VIL1: Extent of ESPs meeting 

wlllvilUli 1 III U»0» L/lolliwlo \LJala lUi iiy.Sy 










School districts 






Number 


Percent 




Had ESP in school year 1987-88 


6.509 


ill 




Planning to implement an ESP in school year 1988-89 or 
1989-90 2.597 


17 




Not olannlna to imDiement an FSP 

1 H V \ hyts^i II III l\J \^ III 1 L^IWI 1 IWI 1 \ Ol 1 




42 




Total 




100 










Table VIK2: School Years ESPs First 














implemented (Data for Fig.10) 


OwlIUUI ytjdio 




Percent of school 
districts 




lQ7Q.Rn anri hafrtro 
l<7/9 Ov dnU UclUic 




5 




1980-82 




4 




1982-84 




8 




1984-00 




25 




1986-88 




57 










Table VIL3: Number of School Oistncts 








and Schools With ESFs by Program 
Criteria {Data for Fig.l 1) 


Criteria 


Districts 


Schools 


1 


6.509 


38.285 




o 
C 


4.304 


27.431 




o 
o 


2,105 


17.794 










Table ViL4: Use of School Teams and 








Written Plans 






Percent of school 
districts with ESPs 




Used School Teams 




89 




Used Written Plans 




71 




Used Soth Teams and Plans 




66 










Table VIL5: Grade Level of Schools With 








ESPs (Data for Fig.13) 




Average percent of schools in 
districts Wi'ih ESPs 




Grade level 


Schools 
with 
ESPs 


Schools 




Elementary 


53 


52 




Intermediate 


18 


18 




High School 


20 


21 




Other Grade Levels 


10 


10 
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Appendix VEt 

Tables Supporting Figures in Report Text and 
Supplementary Tables 



Table VII.6: Percent of Children on Free 
or Reduced-Pnce Lunch Served by ESPs 
(Data for Fig.14) ^ 



Table VII.7: Student Enrollment in 
Districts With ESPs and All U.S. Districts 
(DataforFig.15) 



Table VH.8: Geographic Area of School 
Districts With ESPs and kU U.S. Districts 
(DataforFig.16) 



Percent of childr n 




Percent of school 
districts wKh ESPs 


0 




2 


MO 




17 


11-20 




21 


21-30 




18 


31-40 




17 


41-50 




8 


51-60 




6 


61-/0 




4 


71-80 




3 


81-90 
91-100 




2 





Percenv school districts 


Enrollment 


W.ih ESPs 


All U.S. 
districts 


10.000 + 


8 


4 


2.500-9.999 


29 


18 


1-2,499 


62 


76 




All U.S. 



Primarily central city of MSA 


5 


districts 
4 


MSA bul not primarily central city 
Outside of MSA 


41 


36 



60 
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Appendix Vn 

Tables Supporting Figures in Report Text and 
Supplementary Tables 



Table VIL9: School Characteristics Cited 
by Districts by Program Criteria (Data for 
Fig.17) 



Percent of districts by 
criteria 



School characteristic cited 


1 


2 


3 


Providing strong instructional ieadefship 


92 


92 


96 


Raising staff expectations for student achievement 


92 


92 


94 


Attaining a broadly understood instructional focus 


90 


93 


97 


Monitoring student achievement to evaluate program success 


88 


90 


91 


Emphasizing basic-skills acquisition 


89 


92 


96 


Promoting staff collaboration and coilegiality 


86 


89 


94 


Improving academic learning time 


83 


84 


89 


Improving parental support 


81 


85 


92 


Attaining a safe and orderly school climate 


76 


80 


88 



Table VIL10: School District Involvement 
In ESPs (Data for Fig.18) 



Table VIL11: Source of External 
Assistance to Districts With ESPs (Data 
for Fig.20) 



Table VIL12: Measures of Achievement 
Districts Used to Evaluate Results of 
ESPs (Data for Fig.24 and Related 
Information) 



Percent of school districts 
with ESPs 



Districts in which all schools with ESPs were required to have 
them 



46 



Districts in which some schools with ESPs were required to 
have them 



Districts in which aU schools with ESPs voluntarily chose to 
have them 



50 



Source 



Percent of school 
districts that received 
external assistance 



SEA 



64 



University 



31 



Educational Laboratory 
Other 



36 



31 



Achievement measure 



Percent of school 
districts with ESPs 



Achievement tests 



83 



Nontest measures 



Both achievement test and nontest measures 
Achievement tests only 



75 
66 



Nontest m>easures only 

No achievement tests or nontest measures used 



J7 
8 
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Tables Supporting Figures in Report Text and 
Supplementary Tables 



Tabte VII.13: Student Subgroups by 

Which Dtetricts Disaggregated 

Achievement Test Results (Data for 
rig.25) 




Student subgroup 

Socioeconomic status 



Percent of School Oistricts with ESPs 
Analyzed Analyzed 
regula rly occasionally 



Table VII.14: Student Subgroups by 
Which Districts Disaggregated Nontest 
Measures (Data for Fig.26) 




«^dent subgroup 




Percent of school 
districts with ESPs 


Socioeconomic status 




22 


Ethnicity- 




29 


Sex 




33 


Prior achievement level 




76 




Percent of children 




Percent of school 
districts with ESPs 


0 




3 


MO 




20 


11-20 




23 


21-30 




19 


3140 




14 


41-50 




7 


51-60 




5 


61-70 




3 


71-80 




3 


81-90 




2 


91-100 




a 


^Less than 1 percent. 



Table VII.15: Percent of Children *>n Free 
or Reduced-Price Lunch Served by ESPs 
In Districts That Do Not Disaggregate 
Achievement Data by Student 
Socioeconomic Status (Data for Fig. 27) 
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